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(54) System and method for selecting and accessing portions of information streams from a 
television 



(57) An improved television is capable of demulti- 
plexing, selecting and distributing information streams 
of different types of information to the television and to 
one or more television companion devices (TCDs). The 
television has a television antenna for receiving a broad- 
cast signal having one or more types of information. 
Each information type has one or more segments. The 
television further has a transmitter for transmitting a 
communication signal to one or more television com- 
panion devices. The types of information can include 
any one or more of the following: a video channel, an 
audio channel, a data channel, one or more graphical 
images, and/or one or more animated images. The seg- 
ments can include any one or more of the following: an 
audio clip or audio program, a video segment or video 
program, a still image, animation, graphics, 3D images, 
and/or a stream of text. The television communicates 
with the TCDs via a communication link. When the tel- 



evision communicates with the TCDs, it embeds an ad- 
dress into the message which identifies one or more of 
the TCDs to be the recipient of the message. Information 
streams and segments which are to be cutput by the 
television and/or one or more TCDs are selected by a 
selector that selects two or more segments from one or 
more of the types of information where one or more of 
the segments is displayed, e.g. on the television, and 
one or more of the segments are transmitted to one or 
more television, companion devices. In an alternative 
embodiment, if the automatic selection of information is 
desired, a Profile Subsystem which contains a profile of 
profile items in a database of information of interest to 
the viewer will control selection of one or more of the 
segments if the segments contain one or more profile 
items. 
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Description 

Field of the Invention 

[0001] This invention relates to the area of digital tel- 
evisions. More specifically, this invention relates to se- 
lecting and accessing different portions of an informa- 
tion stream from a digital television. 

Background of the Invention 

[0002] Analog television in the United States and 
elsewhere in the world is one of the most effective 
means of communication and entertainment, yet it has 
undergone relatively little advancement in recent years 
in comparison to other electronic technologies. Its prin- 
cipal capability has been to distribute individual chan- 
nels of moving images and sound to millions of viewers 
around the world. It has not, however, exploited the phe- 
nomenal technical advances of the past twenty-five 
years nor the wealth of information now available elec- 
tronically. We are now on the verge, however, of ena- 
bling these advances with the introduction of digital tel- 
evision. 

[0003] Recently, webTV enabled the browsing of web 
pages and the display of them on a traditional television 
set. The television can be used for web browsing or pro- 
gram viewing, but cannot be used for both operations 
simultaneously. In both cases, the viewing is done in a 
communal manner, i.e. all viewers get the same infor- 
mation. Recently, some broadcasters have coordinated 
the simultaneous broadcasting of a particular program 
and the serving of associated information on the inter- 
net. Once again, this is done in a principally communal 
environment. 

[0004] Soon, digital television will enable the distribu- 
tion of auxiliary information with a video broadcast. With 
this technology, a video signal can be digitized, pack- 
etized and multiplexed with one or more audio signals 
and one or more data channels. The data channels can 
be closely related to the associated video or may be 
completely unrelated. These multiplexed signals are re- 
ferred to as a program. Additionally, many programs can 
be multiplexed into a single Transport Stream. The re- 
sulting signal can be distributed to television viewers 
through either a terrestrial broadcast, a CATV network 
or other network. Once received, the television can de- 
multiplex the video, audio and auxiliary information and 
output it on the TVs screen and/or speakers. 
[0005] Today, selection of the program content to be 
viewed on a television is accomplished by a channel se- 
lector. It selects the VHF or UHF channel which the tel- 
evision will tune to and can often be controlled by a re- 
mote control device. In this capacity, the selector selects 
one video signal and one or more channels of audio. In 
addition, today s VCRs can be programmed to select a 
particular channel at a scheduled time in the future. Tel- 
evisions do not typically have this feature. 



[0006] A television also uses many of the various 
types of wireless technologies currently available (TV 
remote controls, broadcast signals, etc.). The broadcast 
signal is an example of a signal modulated to create a 
s radio frequency signal which can be transmitted through 
the air. The television receives and demodulates this 
signal to create the original baseband signal. Television 
remote controls often modulate a signal on an infrared 
light to transmit information through the air. In this case, 
10 the television receives and demodulates the signal to 
create the original baseband signal. Cable TV is an ex- 
ample of signals which have been modulated to a radio 
frequency and then transported over a coaxial cable. 
Once again, a television demodulates the signal to cre- 
15 ate the original baseband signal. In some cases, cable 
television networks modulate a laser or light emitting di- 
ode to transport the signal over a fibre optic cable. In 
this case, a photo detector is used to enable the recre- 
ation of the original broadcast signal. 
[0007] Originally, a television was defined as a device 
capable of receiving an analog broadcast signal over a 
VHF or UHF channel and selectively displaying one vid- 
eo signal together with its associated audio signal. To- 
day, the signal received can come from a cable TV net- 
work, a terrestrial broadcast, a video tape recorder or 
camera, a computer network or other video source. As 
the television merges with other devices, such as per- 
sonal computers, it will require an even broader defini- 
tion. In this instance, a more appropriate definition is a 
device which receives a signal consisting of one or more 
video channels and one or more types of information 
streams including audio, data, text and graphics and 
which outputs one or more of these streams. 
[0008] The signals output on a television today can 
include video, audio and data. The audio output is typi- 
cally amplified by an audio amplifier and then sent to a 
pair of headphones or one or more speakers. The video 
is most often displayed on a cathode ray tube, but it may 
also be displayed by a projection system or on a flat pan- 
el display. This latter technobgy includes the relatively 
new thin-film transistor displays. Text characters are al- 
so often overlayed on the video to display the time of 
day, selected channel, closed caption text or other in- 
formation. Typically, one video stream, one text stream 
and multiple audio streams are output. 
[0009] Many televisions today contain processors of 
different types: video processors, digital signal proces- 
sors, etc. Televisions are beginning to merge with per- 
sonal computers and contain subsystems that look like 
PCs. This may include a processor IC, static ram, dy- 
namic ram, interrupt handlers and other peripheral de- 
vices normally associated with a computer. Thus, many 
of the applications which PC users have enjoyed will be 
available in the television system, including the capabil- 
ity to exchange messages and play computer games. 
[0010] Digital television provides forthe simultaneous 
broadcast of both a television program and associated 
data. The data can be displayed with or overlayed on 
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the broadcast video. This technology, however, indis- 
criminately forces all viewers to see the same informa- 
tion and only the information chosen by the broadcaster 
Thus, while one viewer of a football game on television 
may be interested in statistics and another viewer is in- s 
terested in the scores of other games, both viewers are 
forced to view the same information. In addition, only 
information chosen by the broadcaster may be ac- 
cessed by the viewer 

[0011] The Internet, alternatively, provides access to io 
vast amounts of information. While Web browsers such 
as WebTV can provide access to this information, they 
do not provide access to information distributed with the 
video broadcast if that information was not obtained 
from the internet They also provide a principally com- is 
munal viewing experience for the accessed information. 
In other words, all people in the room viewing the TV 
see the entire amount of information shown on the 
screen whether that information comes from the broad- 
caster or the Internet. 20 
[0012] Unfortunately, neither of these systems pro- 
vides customized, individual viewing of the auxiliary in- 
formation available. The prior art systems also fail to 
provide simultaneous access to both the digital televi- 
sion broadcast and the Internet. Therefore certain en- 2s 
tertainment and economic value of television broadcast- 
ing is not exploited by the prior art. 
[0013] It is an object of the present invention to pro- 
vide a technique which alleviates the above drawbacks. 

30 

Summary of the Invention 

[0014] According to the present invention we provide 
a television comprising: a television receiver for receiv- 
ing a broadcast signal having one or more types of in- zs 
formation, each information type having one or more 
segments; and a television transmitter for transmitting 
a send signal to one or more television companion de- 
vices. 

[001 5] Also according to the present invention we pro- 40 
vide a television system comprising: a television com- 
prising: a television receiver for receiving a broadcast 
signal having one or more types of information, each in- 
formation type having one or more segments; and a de- 
multiplexer that demultiplexes the broadcast signal into 
one or more of the types of information; a selector that 
selects one or more segments from one or more of the 
types of information; and a television transmitter capa- 
ble of transmitting one or more of the segments to one 
or more television companion devices; one or more tel- so 
evision companion devices (TCDs) comprising: a com- 
munication interface capable of receiving a signal from 
a television set; an output that presents one or more of 
the segments. 

[0016] This invention is an improved television which ss 
is capable of demultiplexing, selecting and distributing 
information streams of different types of information to 
the television and to one or more television companion 



devices (TCDs). The television has a television receiver 
for receiving a broadcast signal having one or more 
types of information. Each information type has one or 
more segments. The television further has a transmitter 
for transmitting a communication signal to one or more 
television companion devices. 

[0017] The types of information can include any one 
or more of the following: a video channel, an audio chan- 
nel, a data channel, one or more graphical images, and/ 
or one or more animated images. The segments can in- 
clude any one or more of the following: an audio clip or 
audio program, a video segment or video program, a 
still image, animation, graphics, 3D images, and/or a 
stream of text. The television communicates with the 
TCDs via a communication link which can be any one 
or more of the following: an infra red link, a radio link, a 
fibre optic link, and a coaxial link. When the television 
communicates with the TCDs, it embeds an address into 
the message which identifies one or more of the TCDs 
to be the recipient of the message. Information streams 
and segments which are to be output by the television 
and/or one or more TCDs are selected by a selector that 
selects two or more segments from one or more of the 
types of information where one or more of the segments 
is displayed, e.g. on the television, and one or more of 
the segments are transmitted to one or more television 
companion devices. 

[0018] In an alternative embodiment, rf the automatic 
selection of information is desired, a Profile Subsystem 
which contains a profile of profile items in a database of 
information of interest to the viewer will control selection 
of one or more of the segments if the segments contain 
one or more profile items. In addition, an Information 
Processor will process the received information streams 
and segments to identify streams of interest to the view- 
ers. In this case, the audio segments are processed by 
any one or more of the following methods: using speech 
recognition to generate a searchable transcript for key 
word searching, audio processing to determine the type 
of audio including any one or more of the following: 
speech, music, noise, sounds, and speaker identifica- 
tion processing to determine the identity of a particular 
speaker. The video segments are processed by any one 
or more of the following methods: using image process- 
ing to identify a searchable person, place, thing, action, 
or event. The data segments are processed by any one 
or more of the following methods: using text processing 
to generate a searchable hit list of one or more words. 
The Information Processor will generate identifiers from 
the processed segments and the identifiers will be used 
by the Profile Subsystem to control the selection of the 
information streams. 

Brief Description of the Drawings 

[0019] The foregoing and other objects, aspects and 
advantages will be better understood from the following 
detailed description of preferred embodiments of the in- 
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vention with reference to the drawings that are include 
the following: 

Figure 1 is a block diagram of an improved televi- 
sion system using a television companion device; 
Figure 2 is a block diagram of an improved televi- 
sion; 

Figure 3 is a block diagram of a preferred television 
companion device; 

Figure 4 is a block diagram of a Profile Subsystem; 
Figure 5 is a flow chart of a selection process; and 
Figure 6 is an example of a profile configured with 
profile items. 

Detailed Description of the Invention 

[0020] With the advent of dig ital television , broadcast- 
ers will no longer be limited to broadcasting a single pro- 
gram of video and audio. They will soon have the ability 
to transmit multiple programs of audio, video, data and 
other information to the viewer. To capitalize on these 
new capabilities, a new television system is needed to 
facilitate the viewing of this information. 
[0021] Figure 1 is a block diagram of a novel television 
system 101 with various embodiments of the television 
companion device (TCD) 120. In this system 101, the 
television 100 presents content to the general viewing 
audience 65 and communicates with a number of tele- 
vision companion devices 1 20 via a communication link 
(109, 1 21 ). In one preferred embodiment, the TCDs 1 20 
are handheld devices used to present individual, cus- 
tomized information to viewers. In alternative embodi- 
ments, e.g. in a teleconferencing room or in a bar, the 
TCDs 120 can be mounted into a terminal, e.g. a table 
or desk, provided to each viewer 65. The goal here is to 
identify, select and view information streams of interest 
to the general viewing audience 65 (television) and for 
each individual 65 (holder/user of a TCD). This will help 
the viewers (e.g. multiple numbers of 65) find and view 
the streams which have the highest entertainment or in- 
formational value to both the general audience 65s and 
the individual viewer 65. 

[0022] The information streams 185 are audio (e.g., 
English and/or Spanish commentary of a soccer game), 
video (e.g., the moving images ol the soccer game), da- 
ta (e.g. statistics on the players), graphic (e.g. graphs 
of the statistics), still image (e.g. , portraits of the players) 
or animation streams (e.g. computer generated videos 
diagramming and explaining a play in the game) broad- 
cast over a television network. In a preferred embodi- 
ment, these information streams, or streams, typically 
185, comprise a plurality of byte packets, typically 188, 
as specified in the MPEG-2 specification. In a preferred 
embodiment, they are time multiplexed together to cre- 
ate a Program Stream, typically 186. One or more Pro- 
gram Streams 186 can then be multiplexed together to 
create a Transport Stream (TS) 191 that is transmitted 
from the broadcaster. (A well known Program Identifier 



(PID) is used to identify each of the information streams 
185 within the TS 1 91 .) In Digital Television, the Trans- 
port Stream 1 91 is broadcast to the viewer 65 on a VHF 
or UHF channel. 
5 [0023] As stated above, some of the important oper- 
ations are the identification, selection and presentation 
of the information streams 185 to the audience (65s). 
Although the television 100 must perform the selection 
for the general viewing audience 65s, the streams of in- 

10 terest 185 to the individual viewers 65 can be selected 
by the television 1 00 and/or the TCDs 1 20. As described 
below, the selection process can be executed automat- 
ically or can require manual intervention by the viewer 
65. Once selected, the streams 185 for the general view- 

is ing audience 65s are output on the television 100. The 
streams 185 selected for the individual viewers 65s are 
transmitted by the television 1 00 to the TCDs 1 20 where 
they are output on a TCD output (125-127), e.g. an op- 
tional display device (126-127) and/or output on an op- 

20 tional audio device (1 25). 

[0024] The information streams 185 which are to be 
output on either the television 100 or TCD 1 20 must first 
be identified and selected. A preferred embodiment of 
this system includes a novel Selector (104 and 124 be- 

25 low) which provides for the intelligent selection of 
streams 185 for a television 100and/orTCDs120. Since 
either the television 100 and/or the TCD 120 may per- 
form the selection, the selector (104/124) can be used 
in either or both devices (100/1 20). In one preferred em- 

30 bodiment, the television 100 and the TCD 120 contain 
a selector (104/124). Thus, the television 100 has the 
option of broadcasting all received streams 185 to all 
the TCDs 120, a subset of the streams 185 to all the 
TCDs 120, a subset of the streams 185 to a subset of 

35 the TCDs 120, or only selected streams of interest 185 
to particular TCDs 1 20. In one preferred embodiment, 
e.g. during a particular time of day, only the TV Selector 
104 selects the selected streams 185 and selects the 
subset of TCDs 120 to receive these selected streams. 

40 For instance, using this embodiment, a parent could limit 
information (stream 185) access by a child using the 
TCD 120. In an alternative embodiment, the TV 100 
sends all streams 185 to all TCDs 120. Here only the 
TCD 120 Selector 124 selects streams of interest 185 

45 to display to each particular user 65 on the respective 
TCD output (125-127). One application for this embod- 
iment would be a public setting, e.g. a bar, with a large 
number of TCD users 65, each interested in specific sta- 
tistics selected from a large number of statistical 

50 streams 1 85 about one or more games being viewed 
communally on the TV 1 00. In still another preferred em- 
bodiment, both the TV 1 00 and one or more of the TCDs 
120 does the selection. In this case, the TV selects sub- 
sets of streams 185 to transmit to one or more subsets 

ss of TCDs 1 20. Then the TCDs 1 20 will perform an addi- 
tional selection of the incoming streams 185 to select 
the exact streams of interest for each particular viewer 
65. In this embodiment, the TV 100 selects basketball 
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game streams 185 and baseball game streams 185 
from all the sports streams 185 received by the TV 100. 
Then each individual TCD 120 user 65 selects which of 
the basketball/baseball games and/or statistics he wish* 
es to view. 

[0025] The communication link between the television 
100 and the TCD 120 can utilize a number of different 
technologies. In some implementations of the improved 
television system, the TV Transmitter 108 and TCD Re- 
ceiver 128 (the communication link) can be radio fre- 
quency transmitters and receivers (with respective TV 
antenna 109 and TCD antenna 121), fibre optic trans- 
mitters and receivers or coaxial drivers and receivers. 
In the former case, the signal will be transmitted through 
the air as in the case of the radio frequency broadcast 
described above. In the latter cases, either a fibre optic 
or coaxial cable will transport the Comm Signal 1 40 from 
the Television 100 to the TCD 120. These technologies 
are well known. 

[0026] Figure 2 is a block diagram of the novel TV 1 00 
and Figure 3 is a block diagram of the Television Com- 
panion Device (TCD) 120 showing more detail. The TV 
100 contains a known Broadcast Antenna 101 used to 
receive a known terrestrial Broadcast Signal 150. In one 
preferred embodiment, the Broadcast Signal 150 is a 
digital signal-which has been RF modulated using a 
known 8 Vestigial Side Band (VSB) modulator and then 
transmitted in the 6 MHz bandwidth of a traditional VHF 
or UHF channel. Typically, each VHF or UHF channel 
will contain one digital signal of up to approximately 20 
Mbps. In an alternative embodiment, there is no anten- 
na; the signal is received over a coaxial cable or other 
transmission technology. This digital signal comprises 
different types of time-multiplexed information streams 
185 including any one or more of the following: one or 
more video streams, one or more audio streams (chan- 
nels) and one or more data streams (channels). Each 
of these types of streams 1 85 can be further divided into 
segments (see Figure 4) where each segment compris- 
es a beginning, middle and end of any of the following: 
a video clip or program, an audio clip or program and a 
data segment. The streams and segments are time mul- 
tiplexed together to create the digital signal 102A. These 
streams and segments are well known. 
[0027] After being received by the Broadcast Antenna 

101 , the signal 150 is demodulated by the TV Receiver 

102. This will recreate 102A the original broadcast dig- 
ital signal comprising time multiplexed video streams, 
audio streams, and/or data streams 185. In high-end tel- 
evisions, the TV Receiver 1 02 will be capable of demod- 
ulating multiple signals, thereby allowing the Television 
100 to simultaneously select information streams 185 
from different broadcast channels. The TV Demultiplex- 
er 103 will demultiplex the video, audio and data 
streams 185 from the digital signal 102A into individual 
segments of video, audio and/or data 109A. Integrated 
circuits are currently available which will multiplex and 
demultiplex the transport streams 191 into a small 



number of individual information streams 185. They uti- 
lize the PID of each stream 185 to identify whether the 
stream 185 is audio, video or data. Some embodiments 
of the improved television system 100 will require a De- 
5 multiplexor 103 which can output a large number of 
streams 185 simultaneously. This is done by cascading 
knowing Demultiplexers using known techniques. 
[0028] Once demultiplexed, the TV Selector 104, if 
used, will then select one or more stream(s) 185 to be 
io optionally output on the TV 100 itself and to be sent to 
one or more TCDs 120 (through a novel TV Multiplexor 
113 (below), TV transmitter 108 and a TV communica- 
tion output, e.g., Antenna 109. The TV Selector 104 is 
a specialized device which is capable of selecting one 
15 or more information streams 1 85 from a multitude of in- 
put information streams 185 received from the TV De- 
multiplexor 1 03. The TV Selector 1 04 is described in de- 
tail below. 

[0029] As the streams 185 are received, the optional 
TV Processor 1 1 2 will search the data within the streams 
185 to locate important information such as the title of 
a video segment, etc. This information can then be dis- 
played by the TV Processor 112 on the TV Screen 107 
to inform viewers 65 of the existence of other information 
streams 185 which they may choose to select. The TV 
Processor 1 1 2 will generate menus which the viewer 65 
can have displayed on either the TV Screen 107 or the 
TCD Screen 127. If the menu is to be displayed on the 
TCD Screen 127, the TV Processor 127 will send the 
menu to the (TCD) Processor 1 34. This send operation 
is described later. As described below, the Menu Pointer 
130 can be used to highlight an item in the menu and 
thereby select an information stream 185 for output on 
either the TV 1 00 or TCD 1 20. 

[0030] The data, audio and/or video streams 1 85 se- 
lected for the Television 1 00 will be sent to the TV Video 
Output 106 for display on the TV Screen 107 (optional). 
In this embodiment, an MPEG-2 video decompression 
IC is used to decompress the video stream and create 
an analog video signal which can be easily displayed 
using known video output devices. The TV Screen 107 
can be a CRT, a flat panel display, a projection display, 
a thin film transistor (TFT) panel or any other type of 
graphical display. The selected audio channel(s) will be 
optionally sent to the known TV Audio Output 105 to be 
output on TV speakers or headphones. The TV Receiv- 
er 102, TV Demultiplexer 1 03, TV Audio Output 105, TV 
Video Output 106, and TV Screen 107 are well known. 
[0031] The TV Selector 1 04 itself is used in a new way 
to provide selected streams and/or segments 109 A to 
the TV 100 and/or the TCDs 120. The Selector 104 will 
receive demultiplexed information streams 185 from the 
TV Demultiplexor 103 on separate well known data 
busses. As the streams 185 are received, the Selector 
104 will be instructed by the TV Processor 112 to select 
particular streams to be output on its separate output 
data busses. In an alternative embodiment, the streams 
185 are selected by a TV Profile Subsystem 110 (be- 
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low). Some of these output busses will send the selected 
information streams 109A to the TV Multiplexor 113 for 
transmission to the TCDs 120. Other output busses will 
send the selected information streams 109A to the 
known TV Audio Output 1 05 and/or the TV Video Output 
106 for output on the TV 100. The method for actually 
selecting the streams 109A of interest is described be- 
low. 

[0032] The TV Multiplexor 113 will multiplex the se- 
lected information streams 1 09Afrom the TV Processor 
112 with information streams 185 from the TV Selector 
104 into Transport Streams 191 which can then be 
transmitted to the TCDs 1 20. The Transport Stream 1 91 
is a well known format and can be found in the MPEG- 
2 Transport specification document. In this embodiment, 
the TV Multiplexor 113 novelly appends an Address 
Header field to the Transport Stream 191 packets 188 
to identify the TCD1 20 which should receive the packets 
188. In one preferred embodiment, each TCD is as- 
signed an address that is used to determine which of 
the packets 1 88 are to be received. A two byte header 
is attached to the beginning of each packet 188. The 
first byte is the address of the TCD 1 20 and the second 
byte is a Linear Redundancy Check (LRC). The LRC is 
well known and is used to detect bit errors which may 
occur in the address. The (TCD) Demu It i plexor 1 23 will 
only demultiplex and output the packet 188 if the Ad- 
dress of the packet matches the address assigned to 
the TCD 120. In an alternative embodiment, the address 
is inserted in the user fields defined in the MPEG-2 
Transport Specification and is utilized in the same way. 
I nan alternative embodiment, the Demultiplexer 123 de- 
multiplexes all Information Streams 185 and the (TCD) 
Selector 124 compares the address of the Packet 188 
to the address assigned to the TCD to determine if the 
stream 1 85 can be selected. 

[0033] In another alternative embodiment, the PID of 
each Information Stream 1 85 is used to determine which 
TCD(s) 120 should receive the Information Stream 185. 
If the stream 185 was selected by the TV 1 00 for a par- 
ticular TCD 120, then the TV Processor 112 will send a 
message to the (TCD) Processor 1 34 containing the PI D 
of the streams 185 which the TCD 120 should receive. 
The Processor 1 34 will then instruct the Selector 1 24 to 
select the stream 185 which has that particular PID. If 
the stream 185 was selected by the TCD 120, then the 
Processor 1 34 will instruct the Selector 1 24 to select the 
stream 185 which has that particular PID. 
[0034] These streams/segments 1 09 A which are des- 
tined for the TCDs 120 will be broadcast via the known 
TV Transmitter 108 through the known TV output, e.g. 
Antenna 1 09. In some implementations of the improved 
television system 101 , the TV Transmitter 1 08 and TCD 
Receiver 1 28 will be Infrared transmitters and receivers, 
fibre optic transmitters and receivers or coaxial drivers 
and receivers. In the former case, the signal will be 
transmitted through the air as in the case of the radio 
frequency broadcast described above. In the latter cas- 



es, either a fibre optic or coaxial cable will transport the 
Communication (Comm) Signal 1 40 from the Television 
100 to the TCD 120. In some implementations, bi-direc- 
tional communications is needed between the TV 100 
5 and the TCD 1 20. In this case, the TV 1 00 will comprise 
a TV TCD Receiver 1 1 4 and the TCD 1 20 will comprise 
a Transmitter 135 utilizing any known communication 
technobgy, e.g. one of the technologies outlined above. 
[0035] In one embodiment, each stream or segment 
10 109A transmitted in this manner will contain an address 
290 which identifies the TCD(s) 1 20 which it is destined 
for. The address 290 can identify all of the TCDs 120 in 
the system, a subset of them or a particular TCD 120. 
If the transmission does not include an address 290, 
is then the Television 100 will broadcast the information 
streams 185 to all the TCDs 120. In one preferred em- 
bodiment, the television 100 will not be able to select 
streams 1 85 for individual TCDs 1 20. The TCDs 1 20 will 
have to either output all the information streams 185 
20 transmitted by the television or it will have to use its own 
Selector 1 24 to select the streams 185 which will be out- 
put. In alternative embodiments, the PID of the informa- 
tion stream 185 can be used to identify streams 185 
which the TV 100 has selected for the TCD 120. 
2S [0036] Each TCD 120 will utilize its known Antenna 
121 to receive the Comm Signal 140 which is transmit- 
ted from the TV 100. The received signal 140 will be 
demodulated by the Receiver 1 28 to recreate the Trans- 
port Stream 191 transmitted from the TV 100. In some 
30 embodiments where the signal is not modulated (e.g. 
some coax link technologies), the signal does not have 
to be demodulated by the Receiver 128. A (TCD) De- 
multiplexor 1 23 will demultiplex the received signal into 
individual video, audio and/or data streams and/or seg- 
35 ments 123A. In some embodiments, the Demultiplexer 
123 is a novel demultiplexer which can utilize the Ad- 
dress Header described above to determine if the infor- 
mation stream 185 transmitted from the TV 100 should 
be received by the TCD 120. It does this by comparing 
40 Us assigned Address to the Address in the Address 
Header which is appended to each packet 188 in the 
Transport Stream 191. 

[0037] Once demultiplexed, the (TCD) Selector 124 
will select which streams 1 85 will be output by the de- 
45 vice. The selected data and video streams 1 24A will be 
sent to a (TCD) output, e.g. a Video Output 126 for dis- 
play on a (TCD) Screen 1 27. The (TCD) Screen 1 27 can 
be a CRT, a flat panel display, a thin film transistor (TFT) 
panel or any type of graphical display. The selected au- 
so dio channel(s) 1 24A also will be sent to a TCD output, 
e.g. the Audio Output 125 to be output on headphones 
or speakers and/or the Video Output 126 after passing 
through a speech recognition system. As described 
above, these are known devices. Note that in alternative 
55 embodiments, one or more of the TCDs 1 20 would have 
only an audio output 125, e.g. if a first set of TCD 120 
users 65 receives audio in a first language while a sec- 
ond set of TCD 1 20 users 65 watching the same TV 1 00 
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display receives audio in a second language. In another 
alternative embodiment, one or more of the TCDs 120 
would only have a Video Output 126. e.g. where the 
TCD 1 20 users 65 are only interested in text information 
that supplements and/or augments the TV 100 video 
that is common to all the TCD 120 users 65. 
[0038] The Selector 124 enables each individual user 
65 of a TCD 1 20 to select parts and subparts (segments) 
of information streams (transport stream, streams, and/ 
or segment) 185 transmitted by the Television 100. Se- 
lection of the streams (segments) (109A and/or 124A) 
to be output by the TCD 1 20 is performed by the Selector 
1 24 and/or the TV selector 1 04 which can be controlled 
in many alternative ways. For example, a Menu Pointer 
130 can be used to select an item from a menu which 
appears on either the TV Screen 107 or the TCD Screen 
1 27. As described above, the TV Processor 1 1 2 will gen- 
erate menus with the names of various streams which 
the user 65 can choose from. Once selected, the TCD 
Processor 1 34 will instruct the Selector 1 24 to output 
the selected stream(s) 185. Alternatively, a message 
can be sent from the TCD Processor 1 34 via the Trans- 
mitter 1 28 to the TV 1 00 instructing the TV Selector 1 04 
to select a particular stream(s) 109A for the TCD 120. 
[0039] The optional Keyboard 129 can also be used 
to identify a stream 185 to be output. The viewer 65 sim- 
ply enters the name of the stream 185. As in the case 
of the Menu Pointer 130, the TCD 120 and/or the TV 
100 will execute the selection 124 of the streams/seg- 
ments. 

[0040] Finally, an optional Profile Subsystem 1 31 and/ 
or an optional TV Profile Subsystem 110 can be config- 
ured to use the selector (124 and/or 104) to select 
streams of interest 185 to the viewer 65. In this case, 
the Selector 1 24 (TV Selector 104) will be automatically 
directed by the Profile Subsystem 131 (TV Profile Sub- 
system 110) to obtain these streams 185. 
[0041] In one preferred embodiment, the Selector 
(104, 124) is controlled by a Profile Subsystem 131 (TV 
Profile Subsystem 110) which contains information of in- 
terest to the viewer 65. This information can include 
broad categories of interest (News, sports, weather), 
types of information (statistics, scores, financial) or spe- 
cific information (person, place, things). The informa- 
tion, formally known as Profile Items (typically 605 be- 
low), is stored in a Profile Database 220. See Figure 4 
below. Each Profile Item 605 contains a category, a key- 
word, and/or keyphrase which is compared to Identifiers 
(201 A, 202A, and 203A in Figure 4 below) which identify 
the received streams 185. Identifiers (201 A, 202A, and 
203 A) can be keywords or key phrases within the infor- 
mation data streams 185 or can be generated by the 
Information Processor (111 and/or 134) described be- 
low. 

[0042] Figure 6 shows an example of a Profile 600 in 
a Profile Subsystem (110 and/or 134). In this embodi- 
ment, the Profile 600 is a database of a plurality of 
records, typically 605, each containing a category field 



615, a Type field 620, and one or more fields with key- 
words and/or phrases 630. In this example, the general 
audience (television viewers) 65s are interested in any 
stream 185 of information related to Fires (entry TV 4). 
s In the category of sports, the users 65s are interested 
in information on any sports person named Pippen (en- 
try TV 1 ) 630, any hockey scores (entry TV 2) 630 and 
any football scores 630 of the Atlanta team(s) (entry TV 
3) 630. The viewer of TCD 1 20 #1 is interested in world 

io news 61 5 for Washington, DC 630 and Iraq 630 (entries 
TCD 1-3 and TCD 1-2) and any political news on Clinton 
630 (entry TCD 1-1). The viewer of TCD 120 #12 is in- 
terested in any events related to festivals 620 (entry 
TCD 1-4), any weather 615 forecast 620 for New York 

15 630 (entry TCD 1 2-1 ) and any information related to Fi- 
nance 615 (entry TCD 12-2). In this embodiment any 
combination of fields can be used to identify an area of 
interest. A Boolean AND operation is performed on the 
profile items 605 when more than one field (615, 620, 

20 630) has an entry. 

[0043] As streams 1 85 are Received 1 28 and Demul- 
tiplexed 1 23, the Profile Subsystem 1 31 and/or TV Pro- 
file Subsystem 110 will search for matches between the 
received information streams 185 and the profile items 

25 605 in the Profile Database 220. The TV Profile Data- 
base 220 contains Profiles 600 for the TV 1 00 itself and, 
optionally, for one or more of the TCDs 1 20. The profiles 
600 for the TV 100 comprises profile items 605 of inter- 
est to the general television viewing audience 65. The 

30 profiles 600 for the TCDs comprise profile items 605 of 
interest to each of the individual viewers 65 in the sys- 
tem. In addition, profile items 605 for a group of individ- 
ual viewers 65s can be described. The Profile Item Com- 
parator 221 will perform the actual comparison as de- 

55 scribed below. 

[0044] In one preferred embodiment, a match of all 
fields ( 615, 620, and 630) in the profile item 605 has to 
occur before there is a match. When a match is found, 
the selector (104 and/or 124) will be instructed to select 

40 the identified stream(s) or segment(s) (109A and/or 
124A) by using the PID to identify the stream 185. As 
described above, the TV 1 00 will transmit the selected 
stream(s) and/or segments 109A to the appropriate 
TCDs 120. In some embodiments, the TCD 120 will 

45 comprise a Profile Subsystem 1 31 which is the same as 
the TV Profile Subsystem 110 but which contains a pro- 
file 600 for the TCD 1 20 only. 

[0045] The optional TV Information Processor 111 will 
process the incoming video, audio and/or data streams 

50 185 in search of information relevant to the viewers. For 
instance, image processing can be used to identify a 
highlight of a football game. If a viewer were interested 
in such an event, the TV Profile Subsystem would be 
configured to indicate this. When the TV Information 

55 Processor 111 identified the event, it would output an 
Identifier (201 A, 202A, 203A) which will be compared 
by the Profile Item Comparator 221 to Profile Items 605 
in the Profile Database 220. If a match is found, the Pro- 
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file Item Comparator 221 will cause the TV Selector 1 04 
to choose the appropriate stream(s) 185 and transmit 
them to the appropriate TCD 120. As described above, 
the TCD 120 would then output the stream(s) 185 for 
the viewer 65. Alternatively, the Information Processor 
132 located in the TCD 120 can process the incoming 
stream(s) 185 and identify stream(s) of interest 185 to 
this particular viewer 65. In this case, the Selector 124 
would be instructed to select the appropriate stream(s) 
185. 

[0046] The TV Information Processor 111 and Infor- 
mation Processor 1 32 can utilize various technologies 
to process and identify streams of interest 185 to the 
viewers 65. This is shown in Figure 4. The Image Proc- 
essor 202 can be used to identify a person, place, thing, 
action, or event which can be compared to the profile 
items 605 in the profiles600. For example, the IBM Que- 
ry by Image Content technology can identify a particular 
color or texture in an image. Similarly, the Audio Proc- 
essor 201 will use speech recognition to generate a 
searchable transcript for key word searching, audio 
processing to determine the type of audio including any 
one or more of the following: speech, music, noise and 
sounds and speaker identification processing to deter- 
mine the identity of a particular speaker I BM s Via Voice 
product, for instance, can generate text from the spoken 
word. Finally, the Text Processor 203 can be used to 
reduce the incoming data information into a searchable 
hit list of one or more words. The Identifiers (201 A, 
202A, 203A) generated by these processors (201 , 202, 
203) are sent to the Profile Item Comparator 221 to be 
compared to items in the Profile Database 220. Each 
Identifier (201 A, 202A, 203A) is compared to each entry 
in the profile 600. If all the entries in a record 605 are 
found in the set of Identifiers (201 A, 202A, 203A) gen- 
erated for a particular Information Stream 1 85, then that 
Information Stream 185 is selected and sent to the TV 
100 or TCD 120 identified in the label of the entry 605. 
A match in this comparison will result in an output on the 
Selector Control 231 signal which will instruct the TV Se- 
lector 104 or Selector 124 to select the appropriate 
stream(s) 185. In an alternative embodiment, the Infor- 
mation Stream 185 will be selected if a subset of the 
Identifiers (201 A, 202A, 203A) for the Information 
Stream 1 85 matches any arbitrary subset of the fields ( 
615, 620, and 630) in the profile item 605. The technol- 
ogies utilized in the TV Information Processor 111 are 
all known. 

[0047] As an example of the improved television sys- 
tem 101, assume three television stations are broad- 
casting on different VHF channels: channels 3, 4 and 9. 
In this scenario, channel 3 is broadcasting a football 
game complete with highlights (NFL) and scores from 
other games in a data stream (SCORES). It is also trans- 
mitting news updates on a separate data channel 
(NEWSUP). Channel 4 is broadcasting a news program 
(NEWS), a news magazine show (NEWSMAG) and a 
documentary on U. S. Presidents (DOC). Channel 9 is 



broadcasting a movie in high -definition format (MOVIE). 
The profile 600 is configured as in the example above. 
There are two TCDs 120 receiving broadcasts from a 
television 100. Each TCD 120 has all of the optional 
s components described above. Figure 5 is a flowchart of 
the processes which occur in the improved television 
system 101. 

[0048] Figure 5 is a flow chart of one preferred selec- 
tion process 500 that is used to performed by the system 

10 101 . This process 500 is now explained by way of ex- 
ample. As the broadcasts are received, the TV Receiver 
102demodulates 501 the three broadcast signals to rec- 
reate the original digital Transport Stream signals 191. 
The TV Demultiplexer 103 will then demultiplex 502 the 

15 signals to recreate the following information streams: 
NFL, SCORES, NEWSUP, NEWS, NEWSMAG, DOC, 
and MOVIE. All of these streams 1 85 will be sent to the 
Selector 104 for selection 503. Simultaneously, the TV 
Information Processor 1 1 1 is processing 51 2 each of the 

20 incoming streams 185 and creating identifiers (201 A, 
202A, 203A) which are used by the Profile Comparator 
221 to determine 51 3 which streams 185 are to be out- 
put (105, 106, 125, and 126). These items are sent to 
the TV Profile Subsystem 110 for use in the selection 

25 process. The TV Processor 1 1 2 is also parsing 51 0 the 
data streams 185 to search for categories 615, types 
620, and/or keywords and keyph rases 630 which are 
indicative of the content of each information stream 1 85. 
In addition, the TV Processor 1 1 2 will also create menus 

30 511 of the categories 615, types 620, and/or keywords 
and keyphrases 630 identified and will display them on 
the Television 100 and TCDs 120 to allow the viewers 
to select streams of interest 1 09A and 1 24 A. 
[0049] In this example, the TV Information Processor 

35 in has generated the Identifier (201A, 202A, 203A) 
FIRE from the news program (NEWS) after it has gen- 
erated the Identifier (201 A, 202A, 203A) ATLANTA 
FOOTBALL from the NFL program (NFL). The TV Proc- 
essor 112 has also discovered the Identifier (201 A, 

40 202A, 203A) CLINTON in the documentary program 
(DOC) and the Identifiers (201 A, 202A, 203A) EVENTS 
FESTIVAL from the movie program (MOVIE). All of 
these Identifiers (201 A, 202A, 203A) keywords are sent 
to the TV Profile Item Comparator 221 . 

45 [0050] The TV Profile Subsystem 1 1 .0 is continuously 
comparing the received Identifiers (201 A, 202A, 203A) 
to the profile items 605 in the TV Profile Database 220. 
After finding a match between the Identifier (201 A, 
202A, 203A) ATLANTA FOOTBALL and Profile 600 en- 

50 try TV 3 (620, 630), the TV Profile Subsystem 110 will 
instruct the TV Selector 104 to select 503 the football 
program (NFL) for the TV 100. Later, when the FIRE 
Identifier (201 A, 202A, 203A) is generated, the TV Proc- 
essor 112 will generate a menu 511 to be displayed on 

55 the TV Screen 107. This will alert the TV 100 viewers 
that a fire is being referred to on the news program 
(NEWS) and will allow them to switch to this program. 
In addition, the TV Profile Subsystem 110 will instruct 
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the TV Selector 104 to select 503 the documentary 
(DOC) for TCD 1 20 #1 and the movie (MOVIE) on chan- 
nel 4 for TCD 120 #12. The streams selected for the 
TCDs 1 20 are sent to the TV Multiplexor 116 where the 
addresses of each of the TCDs 1 20 are appended to the 
streams. The TV Multiplexor 1 1 6 will multiplex 504 these 
streams destined for the TCDs 1 20 together with the 
menus generated 511 by the TV Processor 112. The 
Transport Stream 191 which is generated is then trans- 
mitted 505 to the TCDs 1 20 via the TV Transmitter 1 08 
and the TV Antenna 109. 

[0051] As described previously, the TCDs 120 will re- 
ceive 506 the Comm Signal 140 via the Antenna 121 
and will demodulate 506 the signal to create the original 
Transport Stream 191. The Demultiplexer 123 will de- 
multiplex 507 the Transport Stream 140 into the individ- 
ual information streams 185 and will compare the ad- 
dress of each stream with the address of the TCD 120. 
When a match is found, the Demultiplexer 123 will for- 
ward the stream to the Selector 124. The Selectors 124 
in TCDs 120 #1 and #12 will send the selected 508 in- 
formation stream(s) 185 to the appropriate device (125, 
126) for output 509. Additionally, the Information Proc- 
essor 1 32 and Processor 1 34 in the TCD 1 20 may also 
generate 515 Identifiers (201 A, 202A, 203A) on the in- 
formation streams 185 received from the TV 100. Sim- 
ilar to the operation described above, the Profile Sub- 
system 1 31 will compare Identifiers (201 A, 202A, 203A) 
and will instruct 51 6 the Selector 1 24 to select 508 one 
or more of the information streams 185 for output 509 
on the TCD 1 20 if a stream(s) 1 85 of interest to the view- 
er 65 have been found. 

[0052] Other capabilities enabled by the improved tel- 
evision system include the ability to exchange informa- 
tion between TCDs 120. In this case, messages typed 
on the Keyboard 129 will be input to the Processor 134. 
The Processor 1 34 will format the message, embed the 
address of the TCD(s) 120 which should receive the 
message and transmit the message via the Transmitter 
128. This will allow individuals 65 viewing a television 
broadcast to exchange messages with other viewers 65 
while viewing a television broadcast. 
[0053] In addition, the Processor 134 can run appli- 
cations such as games via the Processor 134. The 
broadcaster is also able to provide auxiliary information 
which can allow the viewer to play along with a game 
show being broadcast. This personalized version of the 
game would allow the viewer 65 to guess answers be- 
fore the results were revealed on the show. 
[0054] The improved TV 1 00 will also provide access 
to the internet to obtain supplemental information. A 
viewer 65 can send an internet request by either typing 
the URL on the Keyboard 129 or by using the Menu 
Pointer 1 30 to select a URL displayed on either the TV 
Screen 1 07 or the TCD Screen 1 27. The Processor 1 34 
will create a request message and transmit it to the TV 
1 00. The TV 1 00 will receive the message as described 
above and will instruct the Internet Interface 115 to re- 



trieve the web pages from the appropriate Internet site. 
Once obtained, the Internet Interface 115 will forward 
the pages to the TV Processor 112. The TV Processor 
112 will either transmit the pages to the TCD 120 for 
s output or will display them on the TV Screen 107. The 
processes for retrieving information from the Internet 
are well known. 



10 Claims 

1. A television comprising: 

a television receiver for receiving a broadcast 
15 signal having one or more types of information, 

each information type having one or more seg- 
ments; and 

a television transmitter for transmitting a send 
signal to one or more television companion de- 
20 vices. 

2. A device, as in claim 1 , where the types of informa- 
tion include any one or more of the following: a video 
channel, an audio channel, a data channel, one or 

25 more graphical images, and one or more animated 
images. 

3. A device, as in claim 1 , where the segments include 
any one or more of the following: an audio clip, a 

30 video segment, a still image, and a stream of text. 

4. A device, as in any preceding claim, where the com- 
munication link is any one or more of the following: 
an infra red link, a radio link, a fibre optic ltnk : and 

35 a coaxial link. 

5. A device, as in any preceding claim, where the out- 
put includes any one or more of the following: a vid- 
eo output, an audio output, and a text output. 

40 

6. A television (TV) comprising: 

a television receiver for receiving a broadcast 
signal having one or more types of information, 
^5 each information type having one or more seg- 

ments; and 

a demultiplexer that demultiplexes the broad- 
cast signal into one or more of the types of in- 
formation; and 

50 a television transmitter capable of transmitting 

one or more of the types of information to one 
or more television companion devices. 

7. A device, as in claim 6, further comprisng: 

55 a selector that selects two or more segments 

from one or more of the types of information, one or 
more of the segments being TVdisplayed segments 
that are displayed on an output of the TV 
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8. A device, as in claim 7, where the television trans- 
mitter is sending one or more segments that are dii- 
ferent than the TV displayed segments. 

9. A device, as in claim 7, further comprising a selector 
input that controls the selector. 

10. A device, as in claim 7, where the selector further 
comprising a Profile Subsystem which contains a 
profile of profile items in a database of information, 
the profile describing interests of one or more 
groups of viewers, each group having one or more 
viewer, and the Profile Subsystem controlling selec- 
tion of one or more of the segments if the segments 
contain one or more profile items. 

11. A device, as in claim 10, further comprising a Proc- 
essor Subsystem having one or more processors 
wh ich process any one or more of the following seg- 
ments before selecting the segments: a video seg- 
ment, an audio segment and data segment. 

12. A device, as in claim 11 , where one or more audio 
segments are processed by using speech recogni- 
tion to generate a searchable transcript for key word 
searching. 

1 3. A device, as in claim 1 1 , that uses audio processing 
to determine the type of audio including any one or 
more of the following: speech, music, noise and 
sounds and using speaker identification processing 
to determine the identity of a particular speaker. 



ments; and 

a demultiplexer that demultiplexes the broad- 
cast signal into one or more of the types of in- 
formation; 

s a selector that selects one or more segments 

from one or more of the types of information; 
and 

a television transmitter capable of transmitting 
one or more of the segments to one or more 
10 television companion devices; 

one or more television companion devices 
(TCDs) comprising: 

is a communication interface capable of receiving 

a signal from a television set; 
an output that presents one or more of the seg- 
ments. 

20 19. A television system, as in claim 16, where the tele- 
vision transmitter identifies one or more of the TCDs 
as a subset of addressees to receive the segments. 

20. A television system, as in claim 18 : that further com - 
25 prises a multiplexer that multiplexes one or more of 

the segments transmitted from the television trans- 
mitter. 

21. A television system, as in claim 20, where the mul- 
30 tiplexer adds one or more addresses to one or more 

of the segments. 



14. A device, as in claim 11, where one or more video 
segments are processed by using image process- 35 
ing to identify a searchable person, place, thing, ac- 
tion, or event. 

15. A device, as in claim 11, where one or more data 
segments are processed by using text processing 40 
to generate a searchable hit list of one or more 
words. 



16. A device, as in claim 11 , where the Processor Sub- 
system generates identifiers from the processed *s 
segments, the identifiers being used by the Profile 
Subsystem to control the selection. 



17. A device, as in claim 11 , where the selector has a 
selector input that is any one or more of the follow- so 
ing: mouse, keyboard, screen, and pointer. 



18. A television system comprising: 
a television comprising: 

a television receiver for receiving a broadcast 
signal having one or more types of information, 
each information type having one or more seg- 
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(57) An improved television is capable of demulti- 
plexing, selecting and distributing information streams 
of different types of information to the television and to 
one or more television companion devices (TCDs). The 
television has a television antenna for receiving a broad- 
cast signal having one or more types of information. 
Each information type has one or more segments. The 
television further has a transmitter for transmitting a 
communication signal to one or more television com- 
panion devices. The types of information can include 
any one or more of the following: a video channel, an 
audio channel, a data channel, one or more graphical 
images, and/or one or more animated images. The seg- 
ments can include any one or more of the following: an 
audio clip or audio program, a video segment or video 
program, a still image, animation, graphics, 3D images, 
and/or a stream of text. The television communicates 
with the TCDs via a communication link. When the tel- 
evision communicates with the TCDs, it embeds an ad- 
dress into the message which identifies one or more of 
the TCDs to be the recipient of the message. Information 
streams and segments which are to be cutput by the 
television and/or one or more TCDs are selected by a 
selector that selects two or more segments from one or 
more of the types of information where one or more of 



the segments is displayed, e.g. on the television, and 
one or more of the segments are transmitted to one or 
more television, companion devices. In an alternative 
embodiment, if the automatic selection of information is 
desired, a Profile Subsystem which contains a profile of 
profile items in a database of information of interest to 
the viewer will control selection of one or more of the 
segments if the segments contain one or more profile 
items. 
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